Ergodicity of Approximate Mcmc Chains with Applications to Large Data Sets
نویسنده
چکیده
In many modern applications, difficulty in evaluating the posterior density makes performing even a single MCMC step slow. This difficulty can be caused by intractable likelihood functions, but also appears for routine problems with large data sets. Many researchers have responded by running approximate versions of MCMC algorithms. In this note, we develop quantitative bounds for showing the ergodicity of these approximate samplers. We then use these bounds to study the bias-variance trade-off of approximate MCMC algorithms. We apply our results to simple versions of recently proposed algorithms, including a variant of the “austerity” framework of Korratikara et al.
منابع مشابه
Perturbation theory for Markov chains via Wasserstein distance
Perturbation theory for Markov chains addresses the question of how small differences in the transition probabilities of Markov chains are reflected in differences between their distributions. We prove powerful and flexible bounds on the distance of the nth step distributions of two Markov chains when one of them satisfies a Wasserstein ergodicity condition. Our work is motivated by the recent ...
متن کاملSpatial Design for Knot Selection in Knot-Based Low-Rank Models
Analysis of large geostatistical data sets, usually, entail the expensive matrix computations. This problem creates challenges in implementing statistical inferences of traditional Bayesian models. In addition,researchers often face with multiple spatial data sets with complex spatial dependence structures that their analysis is difficult. This is a problem for MCMC sampling algorith...
متن کاملDistributed Markov chain Monte Carlo
We consider the design of Markov chain Monte Carlo (MCMC) methods for large-scale, distributed, heterogeneous compute facilities, with a focus on synthesising sample sets across multiple runs performed in parallel. While theory suggests that many independent Markov chains may be run and their samples pooled, the well-known practical problem of quasi-ergodicity, or poor mixing, frustrates this o...
متن کاملMonte Carlo Methods and Bayesian Computation: MCMC
Markov chain Monte Carlo (MCMC) methods use computer simulation of Markov chains in the parameter space. The Markov chains are defined in such a way that the posterior distribution in the given statistical inference problem is the asymptotic distribution. This allows to use ergodic averages to approximate the desired posterior expectations. Several standard approaches to define such Markov chai...
متن کاملExtremal Indices, Geometric Ergodicity of Markov Chains, and Mcmc
We investigate the connections between extremal indices on the one hand and stability of Markov chains on the other hand. Both theories relate to the tail behaviour of stochastic processes, and we find a close link between the extremal index and geometric ergodicity. Our results are illustrated throughout with examples from simple MCMC chains.
متن کامل